1. Field of the Invention
Embodiments herein present a virtual memory technique for efficiently solving connected problems in a distributed environment.
2. Description of the Related Art
There is a class of connected problems that involve problem sizes that span a single machine—common examples are Finite Element Model problems that involve simulating large physical systems. When dividing a problem across multiple machines in order to speed up the overall elapsed computation time, a common technique is to use a set of machines with enough memory that the distributed problem pieces all fit in real memory on their respective machines. When there are not enough machines with enough memory to contain the problem, then some other solution must be used. Presently, the prior art lacks a method for managing the problem pieces when they do not fit in the available memory. Moreover, the prior art presently lacks a method for building an efficient virtual memory mechanism for this grid environment that will work in a general-purpose way for any parallel problem.
Currently, there are no general-purpose solutions to this problem, other than the use of general-purpose Operating System virtual memory. Any application-specific solution is not useful as a general resource for solving parallel problems. Further, any proposed problem is application-specific, or too inefficient.
As large compute jobs become more plentiful, there will be a business in selling compute cycles and compute systems (grids, clusters) to businesses with large applications to run. In some cases, the pool of compute resources will be adequate (there will be one compute node for each compute problem “piece”). However, in other cases there will not be enough compute resources to go around. There must be a way to run these jobs, albeit at a somewhat slower pace, that does not rely on ad hoc solutions or inefficient general-purpose operating support for virtual memory.